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EDITORIAL 


by Brett Stalbaum 


Moore’s Law, Gordon Moore’s famous prediction that 
processing speeds double approximately every 18 months, 
has proven to be so prescient that it long ago rose past the 
status of provocative futurist claim to the level of pedes- 
trian cultural assumption. But what has not yet become an 
accepted cultural assumption is that Moore’s law is at least 
matched, and possibly exceeded by the exponential growth 
of data to be processed. The relationship between 
humankind’s ability to collect data and to process and 
understand data is co-exponential: both are exploding. 
Data sets from genomics, astrophysics, geography, 
geology, particle physics, climatology, meteorology, 
nanotechnology, materials science and even the search 

for ET are producing quantities of data that challenge 

the technical limits of super computers, distributed 
computing, grid computing, and superscalar simulation 
techniques. Even given Moore’s law, optical networks, 
and cheap mass storage, the problem of big data is 
nevertheless looming larger as our ability to collect data 
actively competes with our ability to process and digest 

it. 


Computation has already become a nominal, if not tacit 
assumption in contemporary art practice due to the 
ubiquitous implementation of computer and communica- 
tions technologies in all aspects of our emerging global 
culture. How does big data impinge on the present 
generation of representational artists who operate under 
the assumption of a rich computational environment? And 
what are the emerging aesthetic and conceptual parameters 
that impinge on the practice of artists who consciously 
recognize data and coding as the primary expressions of an 
art practice wherein the notions of “representation” are not 
limited to narrowly prescribed assumptions regarding a 
specifically graphical interface and networked distribution 
as the primary cultural operatives between artist and 
audience? What other questions arise in an environment 
where we live in a constant streaming wash of data, and 
what are the issues surrounding how artists might help 
interpret both cultural and scientific phenomena? 


Lev Manovich raises a particularly interesting issue in his 
2002 essay titled “The Anti-Sublime Ideal in Data Art”. In 
it, Manovich identified an aesthetic approach to big data 
that seeks to interpret large data sets on much the same 
terms as designers and scientists seek to analyze data; a 


pursuit which he describes as the exact opposite goal of 
romantic art. “If Romantic artists thought of certain 
phenomena and effects as un-representable, as some- 
thing which goes beyond the limits of human senses and 
reason, data visualization artists aim at precisely the 
opposite: to map such phenomena into a representation 
whose scale is comparable to the scales of human 
perception and cognition.” He goes on to form a critique 
of such practice, and raises the question of “How new 
media can represent the ambiguity, the otherness, the 
multi-dimensionality of our experience... In short, rather 
than trying hard to pursue the anti-sublime ideal, data 
visualization artists should also not forget that art has the 
unique license to portray human subjectivity — including 
its fundamental new dimension of being ‘immersed in 
data.’”! 


...what are the emerging aesthetic and conceptual 
parameters that impinge on the practice of artists 
who consciously recognize data and coding as the 

primary expressions of an art practice? 


This issue of the YLEM Journal looks to the writings of 
two artists whose practice conspicuously intersects with 
questions relating to the romantic and the sublime. Their 
writings, each in a different manner, suggest possible 
paths toward answering the many issues that have been 
raised by the explosion of, and our immersion in, big 
data. Interestingly, Andrea Polli’s “Atmospherics/ 
Weather Works: Artistic Sonification of Meteorological 
Data” begins with a quotation from the romantic 
American poet Walt Whitman’s “Proud Music of the 
Storm”. Polli is interested in how sonification of large 
data sets differs aesthetically from visualization, and in 
helping a sonic “language or series of languages for 
communicating this mass of data needs to evolve.” Not 
only does Polli’s text clearly describe the types of 
aesthetic choices that were necessary in the sonification 
of the President’s Day Snowstorm and Hurricane Bob 
data, but also reveals a successful example of interaction 
between and artist and scientist(s) to reinforce and 
potentially uncover new knowledge through what she 
claims is a potentially more visceral sonic experience of 
data. 


continued on page 14 
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Wednesday, September 8, 7:30 pm 
McBean Theater, Exploratorium 
3501 Lyon St., San Francisco, CA 94123 


Free, open to the public and wheelchair accessible. 


Since recorded sound became possible, technology has 
allowed sound to evolve beyond its fleeting existence. 
Technology and science have inspired sound artists to 
capture, generate and structure new experiences. Ever- 
evolving, sound art has introduced and combined sound 
and technology with several art forms to revolutionize the 
creative experience. Ambitious works may include sound, 
visual performance and/or installation 


Video Game Sound and Synthpunk 

Artist LX Rudis will talk about his techniques using the 
Oberheim Xpander and provide recorded and live ex- 
amples. Credited as an early “synthpunk” performer, and 
an internationally recognized artist who has worked in all 
areas of development and production for both video game 
products and theater. LX’s clients and collaborators 
include: Atari, Sega, Konami, SCEA [sony], Ultracade, 
George Coates Performance Works, SRL, Drum Machine 
Museum, Negativland, The Units and Winston Tong. 


Sound Art and Technology 

Randy H.Y. Yau, will discuss radical and compelling 
works that have influenced the genre of sound art. Yau is a 
curator, composer, sound artist and designer who has been 
active in the sonic arts since 1993. Yau founded and co- 
curates Activating the Medium, an annual sonic arts 
festival which travels through universities, museums and 
alternative art spaces across California including 
SFMOMA. He also founded Auscultare Research, a record 
label releasing sound works from international artists. He 
has conducted sound art radio programs for over 11 years 
and continues to broadcast on KPFA’s “No Other Radio 
Network.” Since 1999, Yau has served as the Curatorial 
Director of 23five Incorporated, a nonprofit sonic arts 
advocacy group based in San Francisco, and is currently 
its Executive Director. 


Sound and Video 

Artist Scott Arford will discuss the synaesthetia of his 
work in video and sound. Well-known for his sound and 
visual performances, fully immersive multichannel sound 
and video installations, and recently, low frequency 
spacial-acoustic explorations with Randy Yau. 


The latter was with Infrasound, with Randy H.Y. Yau. 
Recently, Arford premiered solo works at SFMOMA in the 
Activating the Medium festival, and performed at the 
Australian Center for the Moving Image for Liquid 
Architecture. He has performed in Peru and The Nether- 
lands, and taught a week-long sound workshop at the 
Center for Contemporary Arts in Kitakyushu, Japan. He 
also lectured last year at the San Francisco Performance 
Cinema Symposium. 


Sound and Natural Phenomena 

Artist Joe Colley will discuss possibilities of expanded 
listening and present a demonstration and listening 
exercise focused on amplified dried ceramic clay absorb- 
ing water. Joe Colley is a self-taught sound artist and 
amateur researcher investigating natural and manmade 
phenomena and their exploitation as source material for 
new forms of expression. His work under the name Crawl 
Unit has been released on numerous record labels world- 
wide. He also founded Povertech Industries, a record label 
releasing sound works from artists around the world. 
Colley has performed live and created installations 
throughout the United States and Japan. 


Sound and Architecture 

Artist Michael Gendreau will discuss his sound composi- 
tions and their correlation between his work in acoustic 
research and music. Gendreau has been a musician since 
age 6. He worked with several group organizations, and by 
age 22 founded, with Suzanne Dycus, Crawling With Tarts 
(1983-1998) He used this group as a medium for fascinat- 
ing experiments in sound perception and composition. 
These final studies included the use of small motors and 
turntable mechanisms, mostly performing with one-off 
transcription discs cast by others in the middle of the last 
century, or cut in Gendreau’s studio using a decrepit lathe. 
More recently, in composition and performance, Gendreau 
has sought to enlarge these later studies, adding physical 
parameters of performance spaces (based on studies in 
physics completed in 1989 and current work as an acousti- 
cian. He works primarily working on low-vibration design 
in buildings and noise impact on optical research appara- 
tus). 


Contact: Torrey Nommesen : torrey@nommmesen.com 
415-445-0196. Full information at: http://www.ylem.org 


The Prospect of the Sublime in Data Visualizations 
by Lisa Jevbratt 


Looking outand up 
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The Polar Sea, Caspar David Friedrich 1823-1824 


In Caspar David Friedrich’s (German, 1774-1840) “The Polar Sea” (1823-24) we look out on an endless inhospitable 
ocean of ice, and a shipwreck - a trace of an attempt to do the impossible, to go “there”, to reach for and understand the 


Starry Sky! 


unbearable void. 


We look up at the starry sky and we sense a fear of not 
comprehending and being engulfed, a fear of the 
unknown, and simultaneously we experience a longing 
for the inaccessible, impenetrable darkness. 


These are the classical visuals of the sublime. Images of 
a sense of grandeur we can’t reach, which we can’t 
penetrate or grasp. It is in the very far distant, it is 
hidden in layers of mist, or made inaccessible by a 
climate not suited for us and it instills a sensation of 
deep fear. Yet we urge for it, we are fascinated and 
attracted by it. 


The Rocky Mountains, Colorado. Picture sixty-three taken with my new camera. 2004. Jevbratt. 


Looking down and in 


2004. We look down. We consume satellite and aerial photography in all its forms; on the web we can access detailed 
satellite and aerial photographs looking down on our houses or whatever we want to surveil from above, we are captur- 
ing mountains far below with our first digital camera, we have the poster of “lights emitted from the earth” on our walls 
(maybe pondering what it says to bypassing intelligences ee 
— gods, aliens and others), and we rely on satellite 

imagery to predict weather and track fires. 


We look in. The genome is mapped and we are trying to 3 x ¢ 
figure out how to look at it. New technologies for looking Soe - 
in towards and inside cells, RNA and nano structures are 2 a a 
rapidly developing, and the methods of making periph- ° : i" 
eral evidence of them and their processes are constantly ‘ : 
refined. We look at our networks that produce data about 
ourselves in sublime quantities. > Fie 
hCAI 
4 tick ss _ - ames . ad 
~~. 


Peripheral evidence: two dimensional polyacrylamide gel. 


Ant-Sublime 


The datasets we are looking at now are of no less dimen- 
sion, vastness and grandeur than the datasets that were the 
subject of the classical sublime; and the sensations of the 
sublime harvested by the romantic artist and others is of 
great interest to us when trying to make sense out of our 
datasets today. However, a quite logical argument against 
the possibility of the sublime acting within data visualiza- 
tion can be made. It has been well formulated by Lev 
Manovich in “The Anti-Sublime Ideal in Data Art”. 


“If Romantic artists thought of certain phenomena and 
effects as un-representable, as something which goes 
beyond the limits of human senses and reason, data 
visualization artists target the exact opposite: to map 
such phenomena into a representation whose scale is 
comparable to the scales of human perception and 
cognition. For instance, Jevbratt’s 1:1 reduces the 
cyberspace — usually imagined as vast and maybe even 
infinite — to a single image that fits within the browser 
frame.” 


The entire Internet with The Web as it existed in 1999 (above) 
and 2002 (below). /:1(2), Interface: Migration, Lisa Jevbratt 
20023 


The reasoning is very clear and it troubled me because I 
instinctively knew that it was wrong — both in making the 
case that data visualization by definition is anti-sublime 
and that my project 1:1 would be a good example of this 
case. 


How then can data visualizations utilize the (or be) 
sublime? Why should they aim to? 


Push, pull 


While the datasets of today are as substantial as the ones 
dealt with in the classical romantic sublime, there is a 
difference in direction and force. 


In the original sublime the force is attraction. The object of 
desire is over there, far away and we want to reach it. We 
want to go there, we are scared and intimidated but our 
longing and effort is ‘towards’. When our force (engine, 
energy, luck) fails the ship stops, it does not get closer. The 
forces of nature push us away - we urge to approach. The 
classical sublime was the extreme tension of not knowing 
and wanting to know; we were attracted by the fact that we 
didn’t know. 


Now, looking in and down the force is reversed. If the 
engine in a plane stops, it approaches the ground; the 
natural force is gravity and we want to stay up and away. 
We are pulled down and respond by retracting. The forces 
of nature pull us down, in - we urge to repel. The sublime 
now is the extreme tension between (hypothetical) 
familiarity - the earth is our home, the cells and DNA are in 
our bodies, the networks are our creation - and a method- 
ological distancing. 


Esthetic decision-making 


In the article “Systems Esthetics’? Jack Burnham wrote 
about the new complex process - or systems - oriented 
society, culture and economics he saw emerging: a new era 
in which systems analysis would be the most relevant 
method for making understandings in any discourse. 
Burnham argues that because we can’t grasp all the details 
of our highly complex systems (economic, cultural, 
technical, etc), we cannot make “rational” decisions within 
them or understand them by analyzing the systems or their 
parts. The way to make decisions within them and to 
understand them is by making more intuitive, “esthetic 
decisions”, a concept he borrows from the economist J. K. 
Galbraith. 


This idea has an intriguing parallel in the philosopher 
Emmanuel Kant’s reasoning about the mobilizing effect the 
sublime has on our organizing abilities. He claims that in 
experiencing the sublime, by facing large amounts of 
information, huge distances and ungraspable quantities, our 
senses and our organizing abilities are mobilized. Contrary 
to what might be believed, we feel empowered, able to 
make decisions, and capable to act. 


Many strategies for aiding people in the task of turning any 
large set of data into knowledge assumes that they should 
be presented less information and fewer options in order to 
be able to make sense out of the data. 


However, humans are capable of sorting through enormous 
amounts of visual information and making sensible and 
complex decisions in a split second, (the ability of driving a 
car is one example). Supported by Kant’s idea I propose that 
under the right circumstances, drawing on sensations of the 
sublime, people can, when faced with huge quantities of 
data, be mobilized to make intuitive understandings of the 
data. Many information visualizations and displays are a 
result of the mistake of compressing the information too 
much and decreasing the amount of information through 
calculations that embody assumptions that are never 
explained. The most common mistake in data visualizations, 
artistic or scientific, is not too much information but too 
little. The “images” of the data landscape are not high 
resolution enough for an esthetic decision to be made. 


Meaning is opportunistic 


Why is low-resolution, highly-compressed data representa- 
tion less meaningful? If it is counteractive to a sublime, why 
is that? How does that sense of awe and “aha” that the fear 
and force of the sublime helps us experience transpire? 


Meaning behaves like a parasite. It is opportunistic, taking 
“immediate advantage, often unethically, of any circum- 
stance of possible benefit” (the definition of opportunistic at 
http://dictionary.com). If meaning in fact is opportunistic, 
and opportunism implies an unethical stance, then it could 
follow that meaning does not thrive in an ethical environ- 
ment. This reasoning is more interesting if one understands 
the term “ethic” as an opposition to “faith”. “Ethic” is a 
stance in which one in any moment is aware of one’s goals 
and choices. One has a plan and a way in which to carry it 
out. “Faith” is a stance in which we let go, were we are 
submerged and surrendered, when we are trustingly 
accepting a “truth”, an emotion or a calling. (At the confer- 
ence “Derrida and The Question of Religion” at UCSB in 
fall of 2003, Derrida mentioned during a discussion between 
him and a presenter the concept of the calling and reflected 
on how that concept is not that different from how animals 
follow traces. This constitutes an interesting point for the 
thoughts in this paper.)° 


Culture then is extremely meaningless because so many 
choices have been made, and nature is extremely meaning- 
ful since no choices have been made. It seems like we strive 
to cut the extremes, the very meaningful and the extremely 
meaningless. To make culture more meaningful we create 
unstable conditions for decision making, 1.e., to reduce the 
number of ready-made choices, we create unpredictable and 
arbitrary events and expressions within it. It is interesting to 
see that younger people are more prone to produce these. 
Quite likely a young mind has more difficulty dealing with 
the burden of meaninglessness, and thus tries to minimize it 


by generating arbitrary signs (such as the expressions, 
fashion and sounds of various subcultures). To make nature 
less meaningful we organize and categorize it and our 
experiences of it. (Of course nature is only void of choices if 
one does not believe in a creating god. In fact the very idea 
of a creationist god could be seen as another attempt to 
decrease the meaningfulness of nature). However, another, 
contradictory reaction to nature might be that our ability to 
perceive meaning is numbed by the loudness of it. Just as 
our retina gets saturated after looking at one color and 
creates a ghost image of the opposite color when we look 
away briefly, nature can (falsely) appear as if completely 
void of meaning. 


The result of this reasoning is that as soon as we are trying 
to make what we experience ethical, i.e. succumb to a plan 
and direction by making deliberate choices, the experience 
and its data decreases in meaning. If we semantically 
categorize and search for meaning, it is as if we try to look 
at the dust on our corneas - we can’t see it unless we stop 
looking at it. Everything becomes meaningless when we 
attempt to “capture” the meaning. In the task of visualizing 
huge datasets this means that we need to avoid making 
assumptions about the meaning of the data in order to allow 
meaning to find an opportunity to occur. Perhaps the answer 
to the question in the beginning of this topic is that we need 
to allow the interplay between the extremes, allowing the 
meaningfulness and the meaningless to happen by not 
attempting to reduce either. 


Identity in the non-intended 


Some years ago a student of mine made an interesting 
discovery in a project he made® . It was Web software that 
returned the result of a search for something on a selection 
of search-engines in the reversed order, 1.e., the most 
relevant, however the search-engines define that, was last on 
the list and the least relevant of the relevant sites was shown 
first on the list. The result was striking. The least relevant 
sites, the ones usually so many clicks away we don’t bother 
to look at them, varied greatly between the different search 
engines. The most relevant results, the ones usually dis- 
played on top, were all the same. 


A similar finding was made some centuries earlier by 
Giovanni Morelli (1874-1876). He sought to find a method 
of determining authorship of paintings and came upon the 
fact that authorship is more detectable in the parts of a 
painting done with less intention - the parts which are not 
significant for the author or the genre in which the painting 
is made, such as earlobes and fingernails. His method is 
now called “The Morelli Method”. In art historian Edgar 
Wind’s words it is interesting that “Personality is found 


where personal effort is the weakest”. 7 


Even more strikingly, what seemed to be true on the Web is 
also true in biology, according to Albert-Laszlo Barabasi in 
his book, Linked: The New Science of Networks’.’ Barabasi 
is doing research on the network structures and linkage 
systems of various fields from computer networks to 
biology. He finds that “For the vast majority of organisms 
the ten most-connected molecules are the same.” (p. 186) 
These highly connected molecules, hubs in Barbasi’s 
terminology, are equivalent to the most relevant pages in a 
web search or the traditionally most “important” features in 
a painting. These are the items, nodes, with the most intent. 
Just as the least relevant web pages are the most dissimilar, 
and the least important features such as earlobes say more 
about the painter, the difference between different organisms 
and the production of their identity lies in the least-con- 
nected, least-used or significant molecules. “[O]nly four 
percent of the molecules appear in all of them. Though the 
hubs are identical, when it comes to the less connected 
molecules, all organisms have their own distinct varieties.” 


(p. 187) 
Via Negativa 


These are all evidences that reality does not show itself to us 
in an expected manner, through intention and expression, 
but it reveals itself to us indirectly in small fragmentary 
pieces. The method of searching out those bits and pieces 
without preconceived notions of what to find has been an 
important method in various mystic traditions, and the term 
Via Negativa, possibly coined by Dionysius the Areopagite, 
a late 5th century mystic, is used to describe it. Via Negativa 
is a method of distancing, of negation, in which we claim or 
pretend to not have any preconceived notions of the systems 
that we are looking at. The method has a lot of similarity 
with artist methodologies (such as Joseph Beuys) and now 
also with some contemporary scientific methods.’ For 
example, the process of harvesting, sequencing and map- 
ping the human genome has been described as that of a 
group of people in a dark room fumbling around not 
knowing what is in the room, how the room looks or what 
they are looking for. Someone bumps into a thing with four 
sharp comers and starts to look for other things with four 
sharp corners. Someone else decides to move along what 
seem to be walls and feel their texture, yet another sits still 
and waits for the others in the room to pass by, taking notes 
on their activities or maybe on their scents. 


The value in Via Negativa for data visualization is that it 
creates that opposing force of not falling into a repelling 
force counteracting the gravity pulling us down. The Via 
Negativa enables the sublime to operate. 


' Walter Koprolin, copyright © 2004 by Walter Koprolin_http://www.astro.univie.ac.at/~exgalak/ 
koprolin/Photo/StarF/Cass_Per_SOmm.html 

? Manovich, Lev, “Data Visualisation as New Abstraction and Anti-Sublime”, 

also: http://www.chairetmetal.com/cm07/manovich-complet.htm 

3 http://jevbratt.com/1_to_1/ 

* Burnham Jack, Artforum, September 1968 

5 Jacques Derrida at “Derrida and the Question of Religion”, UCSB, November 2003 


Epil og 


Juno falling and snacking. Still from Spy Kids 2: The Island of Lost Dreams 


If staying up is our (or others, things or beings) effort, then 
the fall, the ultimate inability to do so, is a trope of interest. 
There are significant falls ranging from literature, such as 
the fall of Alice, the girl in Wonderland, to political, such as 
the fall of the Twin Towers. In “Spy Kids 2: The Island of 
Lost Dreams”, Carmen and Juni, the spy kids, fall into a 
model of the landscape, through the mouth of a volcano. 
Their fall lasts for an extended time, so long that they take 
comfortable positions, eat a snack and discus the possible 
outcomes of their fall. In the end they might not have been 
falling for a very long distance. The model that they are 
falling into has an air vent blowing air sufficiently strong for 
them to be lifted and they might have spent most of the time 
in the illusion of falling. 


In a time period of eight weeks I experienced three events of 
falling substance that for one reason or another seemed to 
have significance. On a dreamlike evening just after sunset 
up on a mountaintop overlooking the beautiful cloud- 
covered southern Californian coastline far from above, a 
shooting star released itself from its usual celestial path, 
where we are accustomed to see it disappear far in the 
distance, and fell towards the earth and us as a real physical 
object on fire landing not too far away from us. A few 
weeks later I spent an hour in my closet attempting to clean 
up after a mold infestation that happened earlier in the year 
when I, more or less simultaneously, heard my neighbor 
flush her toilet and felt a liquid substance on my head. Three 
days into my artist residency in Huddersfield, Yorkshire, 
England, where I’ve taken refuge from the daily duties of 
teaching and meetings, an American bomb fell over 
Yorkshire; whatever the target was, it missed. 


° Gielow Ryan, San Jose State University, 1999 

7 Ginzburg Carlo, “Morelli, Freud and Sherlock Holmes: Clues and Scientific Method” in History 
Workshop Journal, 1980 

§ Barabasi Albert-Laszlo, Linked: The New Science of Networks, Perseus Pub, Cambridge Mass., 2002 
°Gamwell, Lynn, Exploring the Invisible: Art. Science, and the Spiritual, Princeton University Press, 
2002 


Atmospherics/Weather Works: Artistic Sonification of Meteorological Data 


by Andrea Polli 


= 


PROUD music of the storm! 

Blast that careers so free, whistling across the prairies! 
Strong hum of forest tree-tops! Wind of the mountains! 
Personified dim shapes! you hidden orchestras! 

You serenades of phantoms, with instruments alert, 
Blending, with Nature s rhythmus, all the tongues of nations 


Excerpt from Walt Whitman’s “Proud Music of the Storm” ! 
Introduction 

For over ten years, I have been creating art works that translate 
numerical data to sound, from algorithmic compositions 
modeling chaos to live improvisation using video analysis 
systems. Areas of particular interest to my research have been 
modeling human methods of improvisation in interactive 
computer systems and using data sonification to illustrate 
complex information. Visualization is the interpretation of 
scientific data through the visual image, and likewise 
sonfication interprets data through sound. Sonifications can 
help scientific researchers understand data in a different way. 


Since 2001, I have been working on the sonification of 
meteorological data in collaboration with Dr. Glenn Van 
Knowe at MESO, Mesoscale Environmental Simulations and 
Operations <http://www.meso.com> a leading firm in the 
development and application of atmospheric and other geo- 
physical models for research and real-time applications. MESO 
works with the Mesoscale Atmospheric Simulation System 


(MASS) to create a highly detailed simulation of the 
weather based on terrain, initial conditions, and other 
factors. The atmospheric data sets produced by MESO are 
extremely detailed, and although they have a variety of 
visualization tools to interpret the data, much of the data 
represented is not visual in nature (temperature and 
atmospheric pressure for example). Through the project we 
wanted to learn what would happen if the data was 
interpreted sonically. In April 2003, we completed a series 
of multi-channel sonifications of two historical storms, a 
tropical hurricane and a winter snowstorm at five eleva- 
tions as part of a storm sonification project called Atmo- 
spherics/Weather Works. 


The Atmospherics/Weather Works project has three 
primary goals: the development of a software system for 
the creation of sonifications based on meteorological and 
other data to be used in performances and installations, 
live and recorded musical performances, and a web site for 
the presentation and distribution of the recordings and 
software. * 


The first public installation of the project was in April, 
2003 at Engine 27 <http://www.engine27.org>, a non- 
profit organization devoted to the research, creation and 
dissemination of multi-channel sound works in New York 
City. A 16-channel sound installation spatially re-creates 


10 


two historic storms that devastated the New York/Long 
Island area first through data, then through sound. The 
resulting turbulent and evocative compositions allowed 
listeners to experience geographically scaled events on a 
human scale and gain a deeper understanding of some of 
the more unpredictable complex rhythms and melodies of 
nature. 


Why is scientific data so often presented as visual informa- 
tion and much less often presented as sound? One reason 
might have to do with time. A still visual image can be 
scanned over time, allowing a viewer to study various 
aspects of an image. A soundscape or piece of music, 
although it is also temporal, cannot be examined in detail 
without the destructive process of stopping, selecting, and 
replaying various parts. Aspects of the visual image are 
also easily defined by viewers. Specific colors and shapes 
can be described and understood more often than specific 
notes or musical phrases. Specific sounds also can have a 
level of ambiguity. Although some sounds are easily 
identified (like a barking dog or a cat’s meow for example) 
the source of other sounds are not quite as clear. If noise or 
an echo interacts with a sound, it is like looking at a visual 
image wearing glasses that are heavily fogged, making 
recognition more difficult. 


However, unlike a still visual image, music and 
soundscapes are inherently narrative. For example, as I 
listen to footsteps and voices outside my apartment door, I 
can determine that two people are walking up the stairs of 
my apartment building. I can determine approximately 
what floor they are on and even gather a little information 
about their relationship (are they a couple? a mother and 
child? have they been recently arguing or laughing?) In a 
visual image, a photograph of a family for example, unless 
the emotional states of the subjects are highly exaggerated, 
an observer is likely to encounter a certain amount of 
ambiguity in determining the relationships between the 
subjects. 


Can an enhanced narrative and emotional content enhance 
the understanding of meteorological data? Some meteo- 
rologists call themselves “storm hunters”. They travel far 
and wide at considerable physical risk in order to experi- 
ence a hurricane or tornado. Is it because the physical and 
emotional exhilaration enhances the scientist’s understand- 
ing of the storm? The storm hunters would most certainly 
answer in the affirmative. They experience the sound, 
scale, and physical properties of the storm as well as its 
direct effect on the environment. A storm experienced only 
through visualization, whether animated or static, does not 
convey this visceral information. Scientists must use their 
imagination to create a mental image of a storm’s potential 
devastation. A sonic experience of a storm can benefit 


communities beyond the meteorologist’s lab. Ifa scientist 
is alerted by a visceral experience that a storm is likely to 
cause destruction, communities may be more quickly 
notified to prepare a proper response to the storm. 

Our work represents a part of a growing movement in data 
sonification research. In 1997, The Sonification Report 
was prepared for the National Science Foundation by 
members of the International Community for Auditory 
Display ICAD).* This report provides an overview of the 
current status of sonification research and proposes a 
research agenda. Most significantly to us as interdiscipli- 
nary collaborators, the report stressed the need for interdis- 
ciplinary research and interaction. Our project is well- 
suited to sonification according to the findings of ICAD. 
The data sets produced by MASS are extremely large and 
complex, and although there are a variety of visualization 
tools in use to interpret the data, much of the data repre- 
sented is not visual in nature (temperature and atmospheric 
pressure for example). The data represented often portrays 
complex changes over time, an aspect of data particularly 
suited for sonification. 


My personal interest in data sonification is in the artistic 
creation of new languages of data interpretation. As 
individuals and groups are faced with the interpretation of 
more and more large data sets, a language or series of 
languages for communicating this mass of data needs to 
evolve. Data interpreted as sound can communicate 
emotional content, and I am particularly interested in the 
sonification of data related to the atmosphere and the 
weather because of the long history of the weather used as 
a metaphor for emotion in the arts. 


Project Planning 


The project began when I met Dr. Van Knowe in the 
summer of 2001 at the first meeting of Bridges, an 
International Consortium on Collaboration in Art and 
Technology, a joint project of The USC Annenberg Center 
for Communication & The Banff Centre for the Arts New 
Media Institute *. Dr Van Knowe had joined MESO as a 
Senior Research Scientist after 24 years as a meteorologist 
for the Air Force. He was Chief of Meteorology at Rome 
Lab in New York where he directed the meteorological 
aspects of all research and was chief of the modeling and 
simulation development branch for the Air Force’s Combat 
Climatology Center (AFCCC) at Scott AFB, IL. 


Dr. Van Knowe and I brainstormed at that meeting and 
then continued to communicate via email and telephone to 
develop a project plan. After developing a proposal and 
being invited to participate in one of the first spatialized 
sound production residencies at Engine 27 to create a 
storm sonification, we met at MESO to plan the project. 


We wanted to create a spatial sonification of one or more 
storms that occurred in the New York area in the recent 
past in the hopes that some members of the audience 
would remember the specific storms. 


Dr. Van Knowe and Dr. John Zack of MESO suggested we 
try to create a sonification of a major winter snowstorm 
that in 1979 was not foreseen by the existing meteorologi- 
cal models and inspired years of research and development 
into improving the models. The “President’s Day Snow- 
storm” initially formed as a weak wave of surface low 
pressure on a front in the Gulf of Mexico on 18 February 
1979. Since this storm was not predicted by the existing 
meteorological models of the time, a large amount of data 
on this storm was available. 


Later, Dr Van Knowe found a strong tropical Hurricane, 
Hurricane Bob, that passed though the same coastal 
region. We decided to attempt to sonify two storms that 
have a very different physical structure to see if the 


Hurricane Bob 
NOAA-12 
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8:30 A.M. EDT 
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sonifications would yield insight into the nature of these 
two different types of storms. 


Modeling the Storms for Spatialized Sound 


Since the Engine 27 space has a very specific and unusual 
16-channel speaker arrangement, we decided to map each 
speaker to a specific point in space proportional to the area 
spanning from Northern Florida to Northern New York 
State and from the Eastern tip of Massachusetts to Western 
New Jersey with New York City situated near the center. 
Simulated point data was to be modeled for an area of 
approximately 1000km. This area was mapped to the size 
and shape of the Engine 27 space. 


The kind of model output needed for sonification was very 
different from the output formats already in use by MESO 
for visualization. Dr. Van Knowe and his colleagues use 

the Mesoscale Atmospheric Simulation System (MASS) to 
create a highly detailed simulation of the weather based on 
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terrain, initial conditions, and other 
factors. MASS takes real data inputs 
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software to output the data in this 
format. Kenneth T. Waight joined 
MESO in October 1987 after complet- 
ing his Ph.D. in atmospheric science at 
the University of Wyoming. His first 
three years at MESO were spent on a project funded by the 
NASA Marshall Space Flight Center. Dr. Waight relocated 
to MESO’s Troy, New York office in 1990 to assist in the 
development of MESO’s real-time operational mesoscale 
modeling system. 


data files 


Dr. Van Knowe then created a complete model of each 
storm at 5 points of elevation: sea level, approximately 
8500 feet, approximately 18,000 feet, approximately 
35,000 feet, and approximately 60,000 feet (or, the top of 
the atmosphere). Each variable was output every three 
minutes for a 24 hour period of the greatest storm activity. 
The model grid resolution was 10km. Nine variables were 
modeled at this stage, but only six variables were used in 
the final sound compositions: atmospheric pressure, water 
vapor, relative humidity, dew point, temperature, and total 
wind speed. 


Overall model area for the data output 
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Top level of the Max/MSP patch used to run the multichannel system at Engine27 
(patch designed by Matthew Ostrowski) 


Peswie Sues 


After the storms were modeled and the data output, we 
were left with 720 data files of 481 values each and the 
daunting task of translating these numbers into sound. 
Engine 27 master programmer Matthew Ostrowski joined 
us at this stage and he and I worked at the Engine 27 space 
for a period of about four weeks creating a system for 
reading and translating the files to spatialized sound using 
Max/MSP. 


We decided to create a composition of each day’s storm 
activity in full at each of the five elevations. We started by 
simply and directly mapping each variable to the pitch of a 
sound sample of a distinct timbre. We somewhat arbitrarily 
used long tones for temperature and pressure related 
variables and percussive tones for water related variables. 
The bank of sound samples used included vocal sounds, 
sounds created by wind instruments, and environmental 
sounds including the sounds created by various insects. 
The resulting sound compositions were interesting, but 
listeners found it difficult to hear the changes in each 
individual variable. 


We then decided to map the total wind speed to the 
amplitude of the sound. Directly mapping loudness to 
wind speed for every speaker (every geographic point) 
created a dramatic spatialization effect. The fastest wind 


speeds, representing the greatest storm activity, created the 
most sonic activity and excitement. 


However, the combination of timbres was still overwhelm- 
ing to the listener, limiting the listener’s ability to make 
sense of the data. At this point, we decided not to limit the 
number of variables presented through the sonification for 
the sake of the public presentation. Had we been creating 
the sonifications for research only, at this stage we might 
have brought Dr. Van Knowe and his colleagues into the 
space to listen to and compare and contrast sound compo- 
sitions created by single variables. However, there was a 
deadline for a public presentation of the work to a general 
audience and aesthetically we felt that the single variable 
compositions lacked the fullness necessary to engage a 
general audience expecting to hear a musical composition. 


The first aesthetic choice was to translate the atmospheric 
pressure data to a very low frequency sound. In doing so, 
listeners lost the ability to hear a detailed melody line 
describing the pressure changes, but gained a visceral 
sense of the storm. 


Then, we began experimenting with using some of the 
variables as filter variables for sound samples representing 
other variables. Some of the variables in the model were 
highly coupled or inversely related to other variables. We 
created a band-pass filter that filtered a sound representing 
temperature with dew point values and filtered water vapor 
with relative humidity values. We found at this point that 
we needed to choose sounds with a wide spectrum in order 
to hear the filtering most effectively. White noise has the 
widest spectrum, and selecting ‘noisy’ sound samples 
proved the most effective in communicating the data and 
also was the most effective aesthetically due to the 
variation in the resulting sounds. 


The scaling of the data for sonification presented particular 
challenges. Although the overall wind speeds varied with 
elevation levels, we decided to use global scaling for wind 
speed. This created the effect of the compositions building 
and receding in intensity. However, using global scaling 
for variables such as temperature mapped to pitch or water 
vapor mapped to a band pass filter proved to be much less 
dramatic than creating a scaling system for each elevation 
level of each storm since the variables differed widely 
between levels. 


Finally, since the sonifications were to be performed in the 
format of a spatialized sound installation, we developed a 
daily schedule in which various compositions present the 
data sets at the five elevations, moving from ground level 
to the top of the atmosphere. In the installation, each storm 
was performed for approximately 1/2 hour six times each 


day. A storm consisted of six approximately five minute 
compositions presenting all variables at a single elevation 
and one combination of elevations based on the heights of 
the speakers. These compositions were marked by a 
number of ringing bell sounds, marking time and elevation 
like the ringing of church bells. 


Conclusion 


The final compositions were well received by both the 
general and the scientific audiences. Visitors to the 
installation particularly enjoyed remembering where they 
were during Hurricane Bob and the President’s Day 
snowstorm while listening to the sonifications. Some 
audience members found a metaphorical meaning in the 
series of rising elevations, finding the compositions nearer 
to the ground to be more visceral while those compositions 
representing activity closer to the top of the atmosphere 
were felt to be more ethereal and spiritual. 


Dr. Van Knowe was particularly intrigued by the 
spatialization of the sound, and was interested in how the 
wave patterns of the storms were moving in space. The 
sonifications reinforced some known aspects of the 
particular storms. The winter storm was more intense near 
the top of the atmosphere while the hurricane’s fastest 
wind speeds occurred at lower elevations. This change in 
intensity was communicated very clearly through the 
varying degrees of loudness of the compositions. The 
patterns of movement of the tropical hurricane were 
known to be more chaotic than the winter storm, and the 
resulting compositions also reinforced this concept. Most 
listeners found that they could understand more the more 


they listened to the compositions, and there was an overall 
consensus that the work opens up doors for more research 
both in science and the arts. 


' Murphy, Francis. Ed. “Proud Music of the Storm” from Walt Whitman: 
The Complete Poems New York: Viking Press; Reprint edition, 1990. 

? Polli, Andrea, and Van Knowe, Glenn, Atmospherics/Weather Works: 
The Sonification of Meteorological Data. 2003. /studio/atmospherics 

3 Kramer, Gregory et. al, The Sonification Report: Status of the Field and 
Research Agenda, 1997. Available from http://www. icad.org/ 

website V2.0/references/nsf.html 

4 The USC Annenberg Center for Communication & The Banff Centre for 
the Arts New Media Institute, Bridges: International Consortium on 
Collaboration in Art and Technology, 2001. 
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Editorial by Brett Stallbaum, continued from page 3 


In a related article which was originally intended to be 
published alongside the articles in this issue, (see YLEM 
Journal Volume 24 Number 6, May-June 2004), Christina 
McPhee discusses data sonification “Sense of Place and 
Sonic Topologies: Towards a Telemimetic Sublime in the 
Data Landscape”. Polli and McPhee share an interest in 
data sonification and collaboration with science, but it is 
especially interesting that they also meet up in something 
of a rapprochement with the romantic tradition that 
Manovich discusses. Polli’s notion of how data 
sonification might lend to a “physical and emotional 
exhilaration [that] enhances the scientist’s understanding” 
is congruent with McPhee’s notion that “...one may turn a 
gaze to what cannot be ‘seen’. Here we move into a zone 
of the sublime.” In an abstract sense, it is the same matter 
of scale that the romantics faced via exposure to a new, 
often breathtaking landscape (during the period of 
colonialist expansion in previous centuries), that is today 
expressed technically as a matter of scaling systems of 
processing as humanity is faced by the expansion of 
scientific data. The current context reactivates the sublime 
as an issue for contemporary artists working with large 
data sets. 


It should be noted that this rapprochement with the 
romantic and the sublime is in no way a conservative one. 
The sublime, which can also be described as a particularly 
human cognitive response to decision-making circum- 
stances wherein the amount of data overwhelms one’s 
deductive reasoning capabilities, yet under which humans 
are more often than not able to think and act to yield 
successful outcomes, is one of the general capabilities to 
date that has evaded machine intelligence. It seems that the 
prodigious deductive abilities of computational systems 
can not yet simulate the prodigious inferential capabilities 
of the human mind. We have not yet entered the period of 
strong AI predicted in JCR Licklider’s 1960 essay “Hu- 
man-Computer Symbiosis”, but rather we continue to exist 
in the symbiotic phase where “Computing machines can 
do readily, well, and rapidly many things that are difficult 
or impossible for man, and men can do readily and well, 
though not very rapidly, many things that are difficult or 
impossible for computers.”? Big data, as it turns out, is a 
challenge even to this successful symbiosis, and the work 
of both Polli and McPhee can be read as attempts to 
engage the human capability to experience the sublime as 
part of the process of understanding big data. The sublime 
is something that people can participate in readily and 
well, and exploring how that capability might assist the 
human drive to develop and refine knowledge is some- 
thing that artists are presently working through, in practice 
and theory. 


Lisa Jevbratt’s “A Prospect of the Sublime in Data 
Visualizations”, is both an attempt to theorize the contem- 
porary situation regarding artists and the sublime in a 
theorizing mode, and an answer to Manovich’s use of her 
work as an example of the anti-sublime ideal. In her essay, 
she explores the potential for a symbiotic human-machine 
space to be understood via the sublime in terms of a 
“methodological distancing” including the concept of “Via 
Negativa” and a proper appreciation of the opportunistic 
nature of meaning that would allow us to take into account 
(romantic) philosopher Emmanuel Kant’s notion regarding 
the “mobilizing effect the sublime has on our organizing 
abilities.” Jevbratt thinks this would help us avoid “The 
most common mistake in data visualizations...”, that being 
“not too much information but too little, their ‘images’ of 
the data landscape are not high resolution enough for an 
esthetic decision to be made.” 


On behalf of the YLEM Journal's executive editor, Loren 
Means, and the YLEM board, and the journal Scale (http:// 
scale.ucsd.edu) which is publishing the larger collection of 
essays online (with the addition of a republication of my 
earlier essay “Software Development Platforms for Large 
Datasets: Artists at the API’, originally in Leonardo 
Electronic Almanac volume 11, number 5, May 2003 ISSN 
#1071-4391), I hope that these writings will help further 
define the problem of the sublime and big data, and 
stimulate further discussion of the issues and opportunities 
presented to artists by the problem of big data generally. 


' Manovich, Lev The Anti-Sublime Ideal in Data Art, 
(2002) http://www.manovich.net/DOCS/data_art.doc 

? Licklider, JCR, Man-Computer Symbiosis, original 
publication 1960, reprinted in The New Media Reader, ed. 
Noah Wardrip-Fruin and Nick Montfort, MIT Press, 
Cambridge Massachusetts/London England, 2003. (77) 


ERRATUM: 


YLEM Journal Executive Editor Loren Means received this 
error report from Australian science fiction writer and critic 
Damien Broderick: "The caption for the cover art on Mar/ 


Apr 2004 shouldn't read: 'My Conception of Paul 
Anderson's Tao Zero' but rather 'My Conception of Poul 
Anderson's Tau Zero'. Then again, I rather like the idea of 
Tao Zero and might steal it for a story or article title... :)" 
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